Bootstrap Variance Estimates for Neural Networks Regression Models

where f is a non linear continuous function, ( ) dt t t x x , , 1 K = x is a vector of d non stochastic explanatory variables defined on a compact א⊂R, and { } t Z is a stationary noise process with zero mean. The function f in the model (1) can be approximated with a single hidden layer feed-forward neural network; Hornik et al. (1989) have shown that this class of non linear functions can approximate any continuous function uniformly on compact sets, by increasing the size of the hidden layer. In this context, the use of asymptotic results for estimating the standard errors of fitted values, if possible in principle, become soon very difficult and almost impractical in real problems. This motivates increasing interest in resampling techniques (see Tibshirani, 1995; Refenes and Zapranis, 1999 inter alia) as alternative and/or complementary tools to the asymptotic ones. The aim of the paper is to extend some of the common bootstrap proposals to the context of possibly non stationary time series, specified according to the model (1), to estimate the sampling variability of the neural network estimators. We particularly focus on evaluation of the accuracy of the bootstrap estimates based on four different approaches: the residual bootstrap, the moving block bootstrap, the sieve bootstrap and the post-blackening bootstrap.